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ABSTRACT 

Social relations data is an especially useful but 
often neglected means of obtaining information on the effects of 
social processes that are involved in educational inter ^ A 

four-fold table conceptualizes educational interventions and their 
outcomes as consisting of social and^acadeihic interventions and their 
outcomes^ This model characterizes programmatic ihterveht ions as a 
mechanism for changing opportunities and making interaction more or _ 
less convenient. The model is applied to evaluation of two nationwide 
policies that call for social intervehtibh in public education: 
racial desegregation and the maihstreaming of mildly handicapped 
children. In the case of desegregation^ data include numerical levels 
of desegregation, the level of effort^ and the repeated measures of 
social interaction . The outcome noted is the level of integration 
across various settings. The optimum process for developing data on 
social outcomes is sociometric testing. (LP) 
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T^^^^ Evaluation of Social Outeomes 
in Education' 

GAHA LEINHAPDT and SAMUEL LEINHARDT 



IntrddHctidn 

Educational ihler\entions are asaaliy expected to affect the cognitive 
performance of individuals regardless of whether the interveritibri irivdives 
social or cognitive processes: As a consequerce. educational evaluations tend 
to stress the differential impact of cognitive' and scM:ial processes oh cdgniti\e 
butcdmes. The relative neglect of social outcomes, whether purposeful or 
accidental. Is unfortunate because it results in continuing uncertainty regarding 
the utility of akemalive strategies for achieving the social goals of public 
pxjiicies. There are two basic reasons for this siluatibn: (a) well- specified 
models that relate social processes to measurable policy-relevant outcomes are 
rarely proposed; and appropriate measurerhent techniques and analytic tools 
that focus on social variables are inadequate. In this chapter we describe a 
conceptaal framework that can aid in clarifying distinctions between social and 
cognitive outcomes: we explore some possible models for social interventions 
that follow from this frameworlt. and we detail an analytic strategy that can 
improve precision in rricasuririg social outcbrries. We apply this approach to two 

teinhardt was supported by the Leapaing Research and Develbprrierit Cenisf:' with funds, 
supplied in part by the National IrisUluie .>f Education jINIE). United States Department of 
Educaijon. Leinhardi was suppt»rted in part Hy grants from the Natibnal Sck 
(SOC 7^-08841) and the National InMttute for Child Health and l iuman Development ( I ROI HD 
J25Q6-11. The opinions expres'^ed do not necessarily refleci the pbsiitbh bf^^ 
NICHHD. and hb bnicial endorsement should be inferred. Authors* names appear in alphabetical 
order 
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social interventions; fa) racial or ethnic integration: and (6) the mainstreaming 
ot ''special" children. FsnaJ^y, we dt^n ^nstrate the use of the analytic strategy 
through the presentation re^ahs iVom an evaluation of a mainstreaming 
4;xperimenL 

Although this oversimpi' le:^ U'f' --ituation. it Is helpful to conceptualize 
educational interventions and their ct^ncdmitant outcomes as a four-fold table 
consisting of academic and social inters vntidns and outcomes, For example., 
acaderriic manipulations can be designed to alvcr the academic behaviors of 
students (Case I in Figure 1 6. IJ^ asjri the iritrixiucti; of a new curriculum or 
ihstructioria! technique ( Ball 1973). Student social bcha' 'orcan be rrianipulated 
with the purpose of altering acaderriic behavior ( Caaf^ 2). for exarriple. the 
tcariis- games tdlirriamerit.s developed by Slaviri ( 1978) and vJih.ers. Less Often, 
academic behavior is rriariipulated in order to alter student social behavior 
iCasc 3). as. forevample. iri the work of Cohen, Lockheed, arid Lbhmari( 1976) 
on cxpectatidris in desegregatiori contexts. Firially, social behavior cari be 
pn^grarrirriatically altered with the objective of changing the voluntary social 
behaviors of the students (Case 4). Desegregation arid mairistreamirig are 
examples of such interventions. .Although any academic intervention can have 
social cv>riseqUerices arici vice versa, we coriceritrate here on Case 4, social 
IriterNeritions designed to have social consequences. 

Iri gerieral. whcri evaluators investigate outcomes of academic interventions 
they u.sually proceed by administering a hatterv' of achievement (academic) and 
attitude (social) tests (Cooley and Leinhardt 1980: Fisher: Fijby: Nlarhave: 
Cahen. Dishaw. Moore, and Berliner 1978: Stebbins: St Pierre: Proper, 
Anderson, and Creva 1977) bat the discussion of results almost always focuses 
on the achievement tests alone: Clearly: education involves more than academic 
achievement: and load complaints about narrowness and rigidity in evaluating 
programs'asing achievement tests alone are frequently heard. Such approaches 
are. hi>wever. unhkejy to be abandoned (House. Glass. McLean, and Walker 
1978): indeed: wfien aitective or noncognitive outcomes are reported, it is very 
frequently because nothmg could be said about the academic outcomes, and we 
are left with the unsatisfying *' happy: self-confident, but ignorant student" 
syndrome that plagues many evaluations ( Webster 1975). Inadequate attention 
to the social outcom^^s of social interventions raises a serious problem for 
applied social science. 

The apparent neglect of the social consequences of social inter.y..**'^'^s is 
surprising for several reasons. First, it acknowledged that the educational 
process inHuences the development of_social behaviors" and cdrii pete ricies iri 
addition to academic achievement (Cohen el al 1976). Secorid. there is 
growing ev idence that peer processes are an important riiediatdr of acaderiiic 
learning and can. through the estabiishmerit of coriiriiuriicatiori arid support 
> net'vorks. t icnitatc_or rru_strate_ the acquisitidri of both acaderiiic arid rioriaca- 
domic ski. (Bar-Tal 1978). Third, the pheribmeridri of self-imposed segrega- 
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Outcome 



Academic 



Social 



Academic 



Case 1 



Case 3 



Intervention 



Social 



Case 2 



Case 4 



piCURE 16.1. this chart refates academic and social Intenemtons an4 omcomes. 



tion of mmorities within programrriatically desegregated schools implies that it 
is possible for the letter but not the intent of the law to be fulfilled ( Francis and 
Schofield 1980), and, therefore, that procedures are needed that can assure 
greater compliance. 



in the United States there are two nationwide policies that call for systematic 
social interventions in pablic education which are the focus of humerdUs 
evalaations: racial desegregation and the mainstreamirig of rriildly handicapped 
children: These policies are rooted in parallel interpretations of the cdrislitu- 
tionai requirernent for equality of all before the law. Both have led to 
programrnatic manipulations in which personal attributes of children are used as 
a ba^iis for locating them in various educational facilities. In the case of 
deseeregation. the relevant attribute is race, and by extehsibri, ethnicity: in 
rnainstreaming. it is a physical or rriental irripairmeht. Both progranis. now 
established policies, hav^ been the fruit of elaborate and costly historic trends 
and are still a cause of continuing public arid institutional concern. 

The eradication of de jure segregatidri fdlldwirig the 1954 Supreme Court 
decision {Brown v. Board of Education) was, relatively speaking. 
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siraighttbru ai-d:*^ The eradication of de facto segiegaliori, bri the other hand, 
since it involves the developrrierit arid irrijDrerrieritatibri of new arid Uritried social 
mechanisms for overcomirig the riatliral conseqtie rices of established resideritial, 
attitadinal, occupatianal. and iristructiorial patterns, is far more complicated 
and has led to the development of riumerous iriterveritiori strategies. These 
strategies can be designed to achieve various legal or politically specified levels 
of racial balance; They also tend to have varyirig impacts ori the targeted 
individuals as well as on the community. In part different implemeritatiori 
strategies exist because policymakers lack adequate information bri the riature 
i>f their indirect impacts: 

DESKGRHGATION AND INTEGRATION 

Simple desegregation, the physical presence of ditTerent categories of 
mdividuals, can and should be evaluated in a relatively straiglitforward manner; 
For the most part, such physical manipulation is a precondition to social 
integration and equality of opportunity, but it is not the same thing; Physical 
desegregation can occur at a variety of levels; the community, the school 
system, the school building; the grade or class type ( i.e., remedial; vocational, 
eenerai, academic), the classroom, or the instructional unit It is achieved by 
^.mpiy placing mdividuals of a particular type in the presence of individuals of a 
dirterent type. Legal mechanisms exist through which to obtain such mixes. ^ 
However, when we tu-n to issues of integration ( the positive social interaction of 
the relevant groups), we face the problem of changing the voluntary behavior of 
individuals, a prbblerri which is beyond the sphere of mandatory actions. The 
information we need to determine vvhether integration occurs is also signficantly 
rriore complex. It includes contextual information, the relative status of those 
coritexts. the desegregation level in each, and the riature of the intergrdup ties or 
J' ntacts that occur iri each situatidri, 

Regardless of whether the objective of a desegregation attempt is simjDly 
desegregatiori or the rribre elijsive iritegratibri, the evaliJative model that has 
freqUeritly beeri erriplbyed, although corriplex, has not beeri \vell specified. It is 
assurried that the simultariebus physical presence bf rniribrity arid rristjbrity 

'\V cdo not mean to imply thai ii was srmple or easy, WL-jnerely point '">ut thai, once a construct 
In devcIi^fHrd in iav<. there are esiahlished procedures for efTeciirig a specific nuicorrie, 

■'a worthvvhile piece bl e<^'aluative trilorinaiibh vvould be the \ei"y simple d«x:umeriiaiibri of this 
a Kmc For example, in 1980, the city of Pittsburgh was 22*^; hlack: the_s_choo! system was 50% 
black: The top academic high sch(x»I was 25,7% black, ninth grade was 32'S) black, twelfth grade 
'A as 25"«> black, remedial programs were black, acadernic programs were IS'V^ bjack. honors 
pro^rarhs w jre \ 0"n black, and so forth. The acaderriically Ib'w.v.'st high sch»>:>l wa^ 99:S'\, black, 
njnih grade was 99.9"« black, twelfth grade was jOO'Nrj black, remedial programs v^ere 100'^^ blacL 
acade mic programs did not exist, nor did honors programs These nested lists of simple percentages 
arc informative and suggest clear lines of dramatic status ditVcrcntials and areas in which policy 
v^orkers can .ind nre placing their efForts; 
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chiidreri will, throogh some anspeciHed process, lead to an improverheril in the 
academic performance of the minority group while leaving the academic 
pcrforniance of the majority group unchanged. Although there seerris to be little 
iriformatlon to support the former assumption, there is some substantial 
information to support the latter (Crain and Mahard 1977). Does this mean that 
desegregation is a faijure" We would argue that the model is misguided and 
leads to irrelevant evaluations. The concern should focus on the social 
outcomes of desegregation and the mechanisms through which the social ties 
that permit integration to develop. 



CONCEPTUAL MODELS 

Developing conceptual rnodels of the processes that are thought to link 
interventions vvith outcomes is an important, part of an evaluation. By 
conceptuui model we mean simfDly a relational scherna. or graph, in W'hich 
measurable inputs are linked to obser\'able outcomes. Such a rriodel helps the 
evalaator specify the processes through which inputs are expected to a fleet 
target individuals and alter obsenabfe behavior. This activity forces the 
evaluator to be explicit about what aspects of the iriterv'eritiori are expected to 
influence the behavior of target individuals, and to what extent specific 
measures are ac*.!ng as prdxiels. for the elerherits of coriceni (Cooley 1978; 
teinh^.rdt 1978: Leinhardt 1980). When the evaluation fcK:uscs on academ^^^ 
outcomes, the use of such an approach is riot novel. It does seem far less 
accepted, howeve r when social bUtcorries are at issue. It is quite hkei> that two 
problems are involved. One is the paucity of applicable soc[al scien;ce theory 
and the other is the lack of agreemerit on what are observable social 
outcomes. 

SociaLscience theory* cbUld be a vital aid in the constraction of conceptual 
models. The -Ty can suggest the cbmporierits. relationshjps, and directionaiity of 
impacts. Further, it can Heip evaluators recognize when apparently difTeren^^ 
components are iri fact the sarhe. By constructin^g conceptual models, the 
evaluator. in etTect trarislates theorv' into the client's policy-rele^ ant dimen- 
sibris. UnfortUriately. m<^st relevant theoretical social science is devoid of 
OfDeratibrialnrnpllcations. Consequently: the evaluator is in a position of either 
rieglectirig theory altogether and proceeding m an ad hoc fashion, or creating a 
theoretical framework. We choose^ this latter approach, but do so in a general 
fashion iri order to estabhsh ^framework that has broad-based applicability iri 
contexts where behavioral rnanlpulations are designed to yield social out- 
comes. 

In developing the models for social interventions used here, w^e hav^e Utiiized 
what we call an opportunities framework as a theoretical guide. Specifically, we 
assume that social interventions alter individuals* bppbrtUriities for socially 
meaningful encounters. Successful integration of riiirioriiies and the haridi- 
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capped, thus, depends in part on the opportunities for contact provided by the 
context. We distinguish betu'een pussive opportunities in which targets may 
observe interactions and may experience primarily cognitive alterations or 
connrmations. and ucu've opportunities in which the targets may encounter 
( physically or verbally) or interact with the *'others/* The metric For both active 
and passive opportunities involves time as well as quality or significance. We 
are asserting that, for evaluative purposes, social interventions can best be 
understood in terms of the alterations in opportunities which they create for 
target individuals to experience passive and active social encounters which 
would not have occurred or would have been significantly less likely without the 
intervention 

Using the <^pportunities framework, a useful model for the impact of 
desegregation can be specined. The outcome of interest is the level of 
iniegrauon acn>-s, a variety of settings, such as. athletics, subject matter, free 
iime. The inputs of interest are the numerical levels of desegregation, the level 
of elTort or Intervention, and the repeated measures of social iriteractibris The 
pi>int here Is that de>egregation is primarily a social event. The objective is to 
alter the social experience and behavior of individuals arid prbrribte a view of a 
socially moral and ethical society. 

SbeiAt ObTCOMES 

Before discussing specific models of integration we need to clarily what 
mean by a social outcome. We have asserted that social outcomes must be 
conceptualized in terrris of observable social behaviors. Educational inter- 
ventii>ns involve manipulatibris of iridividuais that manifestly alter the oppor- 
tunities they experience for social interaction. Such interaction or its observa- 
tion is presumed to affect the development of relations between individuals. 
Thus, social outcomes can be measured in terms i^f either the actual interactive 
behavior of targeted individuals or the relational ties that exist between them. 

Desegregation arid mairistreamirig create increased opportunitjes for inter- 
action betvveeri member.s of dilTererit groups, that is. between individuals of 
different races or ethnicities, and between individuals ^^ho are handicapped and 
those who are riot. Although usually described categorically, for example, 
as iriteractiori betweeri the races or between special and regular groups, each 
iriteractiori irivolves iridiv.iduais and is associated with interpersonal relations to 
arid from them. Such interactions may have positive or negative impacts on the 
desire of the iriteracting individuals to engage in^similar future interactions with 
the same or similar types of individuals; Observation of altered opportunities, 
actual interaction, and the development of interpersonal ties are straight- 
forward. For example: we can conceptualize interventions that are designed to 
alter expectations, encourage interdependence, and promote physical proximity 
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as aiterripts to Ibstcr iriteractibh by making it more convenieni and even 

necessary. ^ 

Successful integratibri should lead lo stable paiiems of relational ties iha^^ 
indinerent to race or ethnicity, and the ties can serve as an observable outcome. 
As Cohen ( 1975) argues, a successfully integrated c!assro<:>m within a 
desegregated school need not be a mas^of completely afPectively intercon= 
nect^d individuals. Successful integration occurs when contacts between 
members of diHereht social groups are nearly as jikely to lead to_ repeated 
contacts and lasting affective ties as are contacts between members of the same 
social group. In Cohen s ( 1975) words. 

The mcchaniNm of desegregation is niv. intended in create uriiversal lo^^^^^ 
The goal ol ihc desegregation prtnjess .s a rrasoMaWt' degree of social iniegraiion and a lack 
i )t . u e r V c < Ml iYk- I h c r c n> b I ac k > .1 nd ii es. g i ^ e n an object i v c i port ant ;o bi^iK can frust 
t a. h Vxhi'f .It: J Ihu^n lo lUl'h r:hi'r suSHcioniK acII to complete 'he task at hand. v. hunjr it 
be d %.H:;uional task, an educatumaj task, or a political ta.sk 1 p. 273: ertiphasis added|. 

The objective in the case of rriainstreaming is analogoas: No one presumes that 
the handicapped child will be imiversally admired. Hked. or loved 
IS for the child to obtain the same level of integration within the group that could 
be obtained by nbrihandicapped children. Social indifT^rence Is ^^^^ is, 
the probability of interaction between two dissimilar individuals should b^ 
essentially the sarrie as the probability of interac^tion between two similar 
indi\ iduais. Integration for the handicapped child means that he or_she is not 
universally disliked, rejected, or hated simply because of a handicap. Patterns of 
interpersonal ties. thus, become a metric against which to ascertain how 
successful integration has been. y 

Figure !6.2 displays a possible model for Integration which possesses 
observable social outcomes related to programmatically mahipulable oppor- 
tunities. The model contains features that are important but ighbred in rriost 
evaluatic^ns. Racial integration J_I Ij js a consequence of the quantity and quality 
of social contacts between the rnernbers of two groups, A and B ( !Q). The social 
coniact can be observed directly, inferred from the structural analysis of 
sociometric responses or inferred from unlinked selPrepbrts. The quantity and 
quality of social contacts are a function of the temporal dppdrtunity (6) 
(controlled for contexts), the physical arranjemerii or groupings (7), the degree 
to which both groups engage in similarly liked and disliked tasks and to which 
both groups; have publicly and privately shown to have similar success (8Mnd 
fmally. teacher treatment(9) of the individual rherribers of the groups m terms of 
instruction, management, "fairness.** arid physical arrangement of the instruc- 
tional setting. - ^ 

Temporal opportunity and physical arrarigement are in part functions of the 
level of dese^reization (5) and the group structure immediately following 
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desegregation (4). i3roup structure immediately toHowing desegregation is a 
function of the attitudes (3) of both groups (toward themselves, others, and 
desegregation) and the stnictures from which the individijals emerged f 1 txnd 2). 
This mixiels assumes that major interventions designed to jafTect integration 
operate through the elements we have specified. Thus. Cohen's work on 
expectation alteration works on opportunity for contact (6) and task skill 
performance (8j. In the absence of such elTorts. the pattern of quick 
rcsegregation and reduction of social contact— especially prpsbcial contact — is 
well documented (Cohen 1975. Rist 1979. Schoeneld 1978, Schoefleld and 
McGivem 1979). 

What role doe5 such a mcxiel play in an evaluation? The evaluation rnbdel we 
are proposing is designed to assess the impact of desegregation on intergroup 
relations. We are arguing that one rnusi be explicit about what things inflijence 
Mich rclatii>nships. A useful evaUiatiori is brie that prb\ides informaiibn relatirig 
level bf impact to manipulable features of the ediJcatibrial erivirohmerit and also 
indicates vvhich elements have positive but cbjiipen sating features and can be 
ignored. For example, if Cohen and Lbckheed are right, theri simjaly rriariijDiJ- 
lating temporal opportunity arid task success iri the right way cari have proTourid 
corise que rices for social contact arid iritegralibri. cbriseque rices suHlcierit, 
perhaps, to overcbrrie or alter attitudinal states or iriiiial group structures. Ori 
the other harid. if the\ are riot right theri such appealirigly simple rhariipiilalibhs 
are not sufTicierit arid more com pie x rriediatibris are required. 



MAINSTREAMING OF MILDLY HANDICAPPED CHILDREN 

Mainstreaming is another educational inters cntioa possessing a ignificant 
social butcbrrie that has become riatibnal policy in the Uriited States. The policy 
results t rum the passage by. Cbngress bf the ^"Education for All the Handi- 
capiped Children Act*" in 1975 ( PL 94- 1 42). By requiring that all children be 
educated iri the "least restrictive erivirbriment/* this act. in efTect. mandated the 
elirriinatii^n of fDrbgrams iri which mildly haiidicappcd childreri receive separate 
special edUcatii^ri Mainstreaming. like desegregation, involves physically 
locating one gri>up of ^iLiderits. the haridicajDped. in the sarrie educational 
context as another, the rionharidicapped. There are. however, two irripbrtarit 
differences between rriairistreartiirig arid desegregatiori. One iri\b!\es the quite 
small number of children who are to be rriairistrearried. The prbpx.'irtibri of 
minorit) students who arc to be integrated irito Uriited States public schbbis is 
about 30'^) of all childreri. Handicapped childreri rriiake up about 1 2"o i)f all 
children: those that are rnildl\ handicapped arid, therefore, likely to be 
main>treamed. represerit drily about drie-half of this or 6%. The derisity of 
mninstreamcd children is sii low that regular classrbbrris wo\ild cbritairi at most 
iwi) special children arid rareiy more thari drie. Thus, the ddrriiriant social 
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system in mainsircamcd classrooms is likely U) remain that of the normal 
children. 

1-he Ncccmd ditVerence involves the etTccii',e nbsencc of a minority sabcaltare 
of handicappcdness (v^ith the excepticin'ot the hearing-Impaired). In addition, 
the values and attitudes ol ;he scKiety at large are the values and attitudes that 
teachers, pv>Ucv makers, ana parents by and large agree shoajd be those acquired 
by the niamstreamed child; Indeed, proponents of mainstreaming freely admit 
that improved access to these values and attitudes is an important underlying 
rationale for mainstreaming. 

The move toward mainstreaming has both academic and social components, 
and ev aUiafions have evidenced these dual concerns. Evaluations oF academic 
outcomes sn mainstreaming contexts are motivated by two distinct points of 
vow; Gne is the realization b> special educators themselves that the use of 
NCparate racilitic> is not an effective academic mode tor most children v^ith 
special needs (Dunn l968). This leads naturally to a need lor evaluators to 
determine whether children with special needs pertorm at least as well in 
n^.amstream classrooms. The other point of view leads to a focus on the 
academic performance of normal children vv ho share their academic exper- 
iences with a main>treamed special child. Here the concern is that their 
achievement is no poorer as a consequence of the interv entidri. 

Evaluations of the social outcomes of rriainstreaming have been relatively 
ctminr'ti. Since the density of special children in rriairistrearri classrooms is 
>.mall >'n average, the attitudinal and behavioral reaction of the norrhal children 
the insertion of a handicapped child into the classrborri social systerri 
Hecr>mcs a critical issue. If there is only one speciaj child in a class, the 
archetypical situation, lack of social integration irtiplies utter isblatiori. In 
ct)iUrast to the situation in a de>egrcgated school where blacks and whites may 
vc^Iuntarily resegregate and develop parallel ihdependem social systems, a 
socially isolated rriainstrearricd child is eftectively excluded from all peer-based 
social activities arid, as a cbrisequerice, except for the opportunity to obser\e 
normal children, has no dpporturiity to develop :he attitudes, behavioral skills, 
and sc;»cial expectatibris that imrrier.sidri in the normal classroom is supposed to 
prov ide ( Chaires 1966). / 

BecaLi^e of the ease with which isolation can be observed and its evident 
deleieribus cbrisequerice^, numerous procedures have been developed to 
improve the charices a mairistreamed child has to develoo interpersonal ties 
with nbrriial classmates. The design of evaluation strategies for these pro- 
cedures deperids Upbri the conceptual model the evaluator has of the main- 
-streaming prtK^css. 

Our mbvJcl derives from the opportunities framework. As with desegregation: 
the piMnt is to ipecif> in some detail the processes that affect the level of social 
iritegratibri bri ari iridividuUl arid lead to mechanisms for meas^armg these 
' coriditibris. Our model is presented in Figure 16:3, It is ver\' similar to that 
proposed for desegregation. 
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In the rriixlel presented in Figure 16.3, the social integration of a target child 
(11) is ah immediate consequence of prior social contacts; both positive and 
negative, arid classroom social structure ( ib)^SociaI smicture is influenced by 
five aspects of classroom activity. First, the temporal opportunity for interaction 
between the target child and his or her peers (6j can be measured simply by 
Uhirig the amount of total time that they are together; [Obviously, more complex 
estiniates can be made by setting and subgroup^) Second, the physical 
aitarigemerit (7) can include an index of sjmiiarjty ( child has a desk, spacing is 
no more or less isolated, etc:) and appropriateness. Third task similarity and 
perforrriarice (8) should include information on the similarity of the task, the 
significance of it, the visabiiity and success of performance, and the visability 
arid succes.s of the product: Fourth: teacher treatment (9) should include 
estimates of appropriateness: frequency of contact in Important dimensions 
such as academic versns managerial areas, compensating behaviors, and 
rejection or Isolating behaviors: These measures most certainly should no^^t be 
simple estimates of equality: Finally, initial or prior social structure (4) of the 
total group is expected to influence the final or posttreatment structure of the 
group. - - 

Physical arrangements of space (7j and teacher treatment of all childreri (9j 
are also influenced by the initial social structure of the groupl4). Initial social 
structure (4) is inflaenced by individual personal attrib_utes (5) and by a jarge 
collection of attitudes and internalized experiences (3). These attitudes include, 
tor example: attitudes toward handicaps, attitudes toward rriairistrearriirig, arid 
feelings and inforrnation about group structure and one's own role in it. These 
attitudes are In turn influenced by 'prior structure \ I and 25. Orie arrow, the orie 
frorn regular group structure to initial structure, rerriairis uridiscussed. We feel 
that in rnany mainstrekmlng situations in which the special child is "dropped 
Into** an existing group, group structure irifluences the effect of mairistreamirig. 
The target chijd*s previous expenences operate thorugh his or her attitudes, 
only, rather than directly on structure. 



DATA AND .MEASUREMENT 

The typical approach to the develoiDmerit of data ori social outcomes involves 
the administration of attitude surveys or other devices designed to determine the 
sentiments or ideas individuals pwDssess about various social groups: The 
underlying assumption is, of course, that such attitudes influence behavior: But 
there are several problems with this approach. First, the relationshjp between 
attitudes and behavior is pborl\ uriderstddd. Second, it is demonstrably difficult 
to change attitudes. Third, the focus of jtiOSt socia| interventions is the 
modification of behavior, riot attitudes. If extant interpersonal ties and 
inte.^actidris are the focus, then the data of interest must reflect this concern. 



16. TUf lEvaiuanoH of Social Outcomes in Educaiioh 

There are two options; one involves gatherjng data that represent observations 
of actual behavior Unfortunately, this requires costly, time-cbrisutning pro- 
cedures possessing significant data summarization, analysis, arid generalization 
problerris. Although such procedures do yield an urimatched richness ot detail 
and are exceptionally well- suited to providing a context-setting perspective, m 
most situations their defects outv^eigh their advantages. The second option is to 

focus on the network of interpersonal ties. _ __ 

The procedure that seems ideally suited to the generation of data on 
interpersonal relations is sociometric testing. Originated by Moreno (1934), 
sdciometric data are, relatively speaking, easily obtained, intuitively compre- 
hensible, and possessed of a high degree of concept validity. In some respects, 
the sociometric instrument is simply a survey of attitudes. In contrast to a 
traditional attitude survey, however, it probes attitudes toward specific 
members of the target population, individuals who actually participate in the 
daily interactions that characterize integration. Further, these relational data 
taken together represent an observation dri the social system: that is, they are 
the measures of mterdependen:L ties that kriit the group members together into a 
coherent social organization. The structural characteristics of these ties are of 
focal interest because they define the locat^ri of the individuals in the ^oup s 
social system. Such characteristics carindt observed in the isolated reports of 
the individuais because they are riot properties of the individuals. They are 
patterns*, not attributes, and can drily be detected by examining the way the 
interpersonal relationships fit together This is a critical point and requires some 

elaboration. . ' 

Clearly, the willingness of childfert of one category to cite children of another 
category as their friends can be thdught of in and of itself as a relevant social 
outcome But a deeper view focuses on consistent patterns of joint citation, that 
iU instances in which specific individuals in different categories cite one another 
as friends. Such outcomes are indications of the mutual trust and openness to 
communicatLori regarded by Cohen and others as essential features of racial 
integration. Similarly, the willingness of mainstreamed children to cite normal 
children as frierids is riot relevant Given the reality of a normal classroom, such 
an outcome is a foregorie conclusion and, given the low derisity of special 
children iri the classroom, their lack of attraction is also somewhat expected. Ari 
esseritial issue, however, is the establistanent of mutual fi-ieridships by the 
majristreamed child, that is, the development of patterns of systematic 
reciprocal involverttent in the socia[ network. 

Socidmetric data are relevant to Sese issues. Methods for collectirig them are 
well kridwri and their use in research on desegregation arid rriairistreaLming is not 
ridvel. Wdst applications have, however, bden methbddlogically cdrtstrained 
arid are not easily generalized. Both the approach to data cdilectidri and the use 
of irieffective anabatic procedures have been at fault (Hdllarid and Leinhardt 
1973). We propose an approach that derives from the riotidri of the opportunity 



15 



308 



Gaea Leinhardi and Samuel Leinhardi 



framework and exploits some new developments in the theory of stochastic 
digraphs: 

Oor approach characterizes the educational interv'ention as a mechanism for 
manipalating opportanities for contact to occur between individuals. The 
interactions that take place between individuals may provide them with new 
informatjon. cause them to change their attitudes and expectations, or otherwise 
^l^^^^J^ modincation in how each perceives the other. Extant ties to other 
members of the^oap rnay» through felt structural imF>erat!ves, also influence 
what is felt to be the consequence of the interaction. Regardless of the mode of 
operation, the interaction results in a relationship between the interacting; 
individuals and it is the relationship; not the interaction, which persists and 
conditions the prospects of future interactions given the opportunities that get 
presented. 

What we are saying, in efieci, is that individuals make choices about whom to 
spend their time with, whom to have as friends. They cannot interact with 
ever>'one all the time. They must choose between competing options given the 
opportunities that the classroom schedule and management structure provide: 
Interaction and observation can operate to modify their priorities and can lead 
them to reevaluate the benefits of repetitions: Programmatic interventions can 
change opportunities and make interaction more or less convenient, but the 
relationships that develop as a consequence of the experience wiU be the 
dominant future factor influencing voluntary social behavior given consistency 
in context. 

The next step in developing this approach requires that we Hnk it to a 
statistical p)rocedure for mckieling friendship choice that relates to relevant 
social outcdrhes and can be used with sociometric data The dlfriculty in 
.accpmpDiishing this step rests in the unique features of relational data and our 
desire to focus on patterns of relations, not distributions of attributes. Recent 
advances in the development of stochastic models for relational data have 
solved many of the problems associated with the analysis of reciprocation in the 
structure of sociometric data. The statistical developrrieht detailed in Holland 
and Leinhardi (1981 ) is quite complex and, consequently, we will hot repeat it 
here. In essence, the theory conceptualizes sociometric data as observations oh 
a systern of affective ties which result from a "choice process/' one ih Which 
individuals allocate their choices to others in a probabilistic fashion^^hereas 
biases act to increase or decrease the probabihjy that choices will ^_to specific 
others over tirrie (Holland and Leinhardt 1977a, 1977b, and 1977c). These 
biases can be based on individual attributes sUch as race or ethnicity, or oh 
structural feature of the social systertl such as reciprocity' arid everi traris- 



In this approach, reciprocity is represented as a bias that acts to iricrease or 
decrease the chance that, once a choice is made, the chosen individual will 
reciprocate the choice. Although deriving from a dynamic view, the approach is 
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eqaally valid at the cross-sectional level. Here the recipiocity bias can be 
conceived of as an Insiantariebus force or effect. A positive reciprocity effect 
means that the observation of a choice of bile individual by another increases 
the chance that, in the same data matrix, the chosen individaa! will be obsei^ed 
to reciprocate. Application. to the cross-sectional case involves specifying a 
model that is a member of a family of exponential probabiji^ty distrib 
defined on digraphs (square matrices comaining entries that ffl-e eilh^ 
ones) and using an iterative algorithni to estimate the parameters of the model.. 
(Programs for fitting these models are available from the authors.) It is 
important to understand that the method developed in Holland and Leinhardt 
( 1 98 1 ) is both multivariate arid parametric. The maltivariate feature means that 
the estimated effect of a bias like reciprocity is made conditional on the impact 
of other model-specified biases, including the attractiveness ( sociometric status) 
or productivity (gregaribusriess) of each individual. The fact that the effects are 
estimates of the parari^eters bf a model means that they can be compared across 
different samples arid that the parameter values can be used as measures or data 

in s^econdary* arialyses. . 

The procedures that Holland and Leinhardt (1981) introduce permit analyses 
of important structural patterns in sociometric data. It is natural to ask how 
these statistical models cart be used to evahiate the social butcoriies bf 
mainstreamirig and desegregation. We pursue an application to mairistreaming 
in depth later, but first we discuss briefly and nontechnically how brie might 

proceed. _ _ _ 

The analytic framework permit^ the investigator to sfjecify alternative 
statistical riibdels for the sociometric data. Extensions permit disaggregating 
elfects tb suit the needs of a hypothesized social outcoriie. For example, an 
effect bf desegregation might be specified in terms of altering the probability of 
reciprocal choice between different groups while hayirig rib iriipact on the 
chances of reciprocal choosing within each group. The situation is slightiy 
modified in the mainstreaming situation since the groups here typically involve 
a single handicaiDped child and a large group of nonhandicapped children. If the 
behavior of the special child is the focus of an evaluative study, the analysis cffli 
examine whether, for example, the probability of a choice is going to the special 
child has increased given that there is a choice coriiirig frbm the special child. It 
is the ability to specify the exact nature of the relatibrial pattern and to estimate 
the effect that render this approach appealirig. 



' DETAILED APPtlCATieisl to MAINSTREAMING DATA 

We have used this arialytic frariiework iri evaluating the social outcomes of a 
mainstreaming expeririierit. This effort is still underway and the results we have 
to date are preiimiriary arid liriiited. Nbrietheless, we present them here as an 
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example of the way in which the opportunities frainework and stochastic 
reiational data analysis procedures can be employed in evaluative. studies. 

We have obtained sociometric data generated during an Office of Education- 
sponsored mainstreaming experiment called Project Priihe which was_ per- 
formed in Texas, circa 1970 (Ballard, Corrrian, Gottlieb, Kaufman 1977)._ 
Project Prime to'^k place ov ^r a period of 3 years and involved a series of 
separate studies. The particular research activity we are concerned with was 
designed to study the integration of mairistreamed educable mentally retarded 
(EMR) children. It has often been observed that mairistreamed EMR childreri 
experierice deleterious social effects. Although they may have beeri socially 
integrated iri their special _educatibri classrdpriis, within the jnairistrearii 
classroom they experierice affective rejection arid social isolatiori (Corrriari arid 
Gottlieb 1978). As iri the case of desegregatibri^ this voluritaristic resegregatiori 
suggests that the affected child riiight*be acaderiiically arid socially better off in a 
programmatic ally segregated classrdbrii. Since the main streaming of EMR 
childreri usually irivblves placing only brie or a sriiall riuriiber of special childreri 
into regular classrbbriil, exploration of secondary interventions that would 
alleviate br eradicate the negative butcbriies of mainstreaming would seerii 
easily accoriiplished. Irideed, riuriierbiis strategies have been developed and 
iriij3leriierited iri the field. Unfortunately, there is little in reported analyses to 
suggest that the systematic isolatiori of riiairistreamed EMR children by their 
classmates can be avoided. 

The cbriipbnent of Project Prime with which we are concerned was designed 
to obtain conclusive evidence that secondary interventions could work. The 
research design used was that of a traditional experimental study: Approxi- 
mately 40 elementary school classrooms, each containing one mainstreamed 
EMR child, were split into experimental and control groups: The experimental 
groups experienced a treatment in the fonrT of a teacher-led group activity 
designed to increase the opportunities for encounters between the teacher, the 
special child, and the normal children, and to increase the opportunities for 
normal children to observe the special child successfully performing a socially 
important task: 

It must be remembered that although the behavior of the mainstreamed child - 
is expected to be affected by the interventions, it is the reactive behavior of the 
normal children that is the focus of the research. The objective is to get the 
normal children to accept the handicapped child, to view the handicapped child 
as much as possible in the same light as they view one another, and to motivate 
the development of reciprocity by stimulating normal childreri to respond to the 
attempts of the handicapped child to initiate interpersonal ties. Iri this particular 
context, it Is explicitly recognized that the social position of the special child is a 
consequence of t^^ norrrial children failirig to accept the special child arid make 
him or her a "regular*' member of their social system. 
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FIGURE 16.4. ThiF is a diagram of the results dftlie Prtme experiment. 



Figure 16.4 presents the two processes dlagramrriaticany. In the cdntrdl 
situatidrt the usual social events occur. The normal child islhbught to pdssess 
negative attitudes and expectations about EMR children. The mainstreamed 
child, on the other handt is'quite positively predisixjsed toward other children, 
handicapped or nonhandicapped: Mainstreaniing creates opportunities for 
actual interaction. When encounters occur they cdhfirrh the childfen's pre- 
conceivadjiotions. The children pa^ with the rnainstreamed child liking the 
nonhandicapped chifd and wanting to engage in Future interactidri, whereas the 
nonhandicapped child rejects the mainstreamed child and prefers td avdid fiiture 
interaction: : 
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Under the ireacmeni condition, the initial situation is identical. The teacher 
whom both hke, however: steps in and, through management of a common 
activity, operates so as to alter the attitude of the nonhandicapped child and 
mediates the children's interaction. The experience is supposed to be positive. 
The resalt should be a change in attitude in the nonhandicapped child. This 
positive relationship is readily reciprocated and leads to future encounters 
which maintain the affective ties. Although not explicitly stated^ the assumi> 
tions in this representation of the process seem to be those made by others who 
have studied the Project Prime data. The results that these investigators 
obtained, however, indicate that the treatmc it had ho noticeable positive 
effect. 

Detailed analyses of the Project Prime sbciometric data will be reported 
elsewhere. We report here two general results. First we investigate whether, 
empirically, there is any benefit accruing from the use of the multivariate 
stochastic procedure as opfDosed to a traditional univariate approach. Second, 
we present some results relevant tb_e_vaiuating the impact of the experimental 
manifDulation on the mainstreamed EMR children. 

The traditional appDroach to the study of sbciometric position focuses oh 
••socionietric status" riieasured siriiply as the number of {or some ribrmed 
function oil choices each group riieriiber attracts. Such measures are the<> 
retically inadequate because they are, in effect, zerb^brder measurements and 
fail to control for the siriiultaneous iriipacts of other relevarit features of the 
system of ties. It is. of course, brie thih^ to have a thebretical rationale behind a 
complex alternative and another to deriibristrate empirical differences in the 
measures, that is. to show that the siriiple measure arid the complex measure 
give dilTerent results. For each gj'bup, we computed chi-sqliafe statistics for a 
hierarchy of models that iricluded a univariate approach to the measurement of 
socibmetric status arid multivariate alternates that contained variables for group 
recijDrocity and choice density, arid individual productivity and attractiveness^ 
First the multivariate models that were fitted to the data were nearly a]ways 
statistically sigriificarit indicating that a Univariate analysis, such as is usually 
p>erfonned iri the maihstreaming literature, was generally inadequate and woujd , 
yield bia.sed results. This finding suggests that prior reports of analyses, which 
foUrid no effect for the experimentaj manipulation, would be erroneous. 

We also contrasted the individual parameter estimates obtained under the 
multivariate approach with those obtained using a univariate approach. The 
multivariate parameter values ranged over a greater number of values than did 
those obtained using a univariate approach. <3ne interpretation of this finding is 
that the multivariate approach makes finer, more precise distinctions between 
the structural features of the group members. 

Finally, the individual parameter values for attractiveness (sbciometric' 
status) and productiveness (gregariousness) of ties obtained using the multi- 
variate approach were plotted against those obtained from a univariate model. 
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These plots re\ealed that the muiiivariale paiarneiers were not monoionic 
runciions of the univariate estimates. This indicates that even simple con- 
clusions drawn, say, on the basis of a rank ordering of the univariate eFTects 
would not necessarily be robust and should not be ased to draw evaluative 

conclusions. _ - 

These three results taken together form a convincing argument for the use of 
the multivariate approach in developing measures of individual social ou^ 
that can be used to evaluate an intervention quantitatively. Indeed, on '-e basis 
of our admittedly limited empirical results, relying on a univariate sta^i.^uc such 
uS •'sociometric status" (or any simple version of this statistic) as an indicator of 
social integration or interpersonal attractiveness is^ dangerous. In the case of the 
Project PrimSii data, it is clear that such reliance would lead to incorrect 
conclusions. In a recent reporuin fact: researchers using univariate measures 
concluded that the Project Prime manipulation had no etTect ( Semme'l, Gottlieb, 
and Rbbinsdri 1979^ As we will show; however, an effect can be associated 
with the experimental manipulation: We discuss this result next. _ 

We evaluated the effectiveness of the experimental manipulation jn the 
following way. One of the planned objectives of the rnanipulatiori was to 
improve the social integration of the mainstreamed child. The opporturiities 
framework led us to conceptaalize this process as a potential influence on the 
attractiveness of the mainstreamed child and as an agent modifying the 
reciprocal nature of the social relations among mainstreamed and regular 
children. Here we report results that focus on attraction. _ _ 

A full multi.i.;iate model was fitted to each sociomatrix. This produced 
parameter estimates for overall reciprocity and density, arid individual 
production and attraction of ties. We extracted the estimates of the iridividual 
attraction parameters for^ach mainstreamed EMR child. We theri used least, 
squares regression to estimate an equation in which pbsttreatrfierit attraction of 
the EMR child was regressed on their pretreatmerit attraction, their gender, and 
the kind of treatment (experimental or control) they experienced. Thus, the 
estimated model was: 

attractiori (post) = attraction ( pre) + treatment + gender. 

This is, in effect, a covariance adjusted analysis in which pretreatment 
attraction is controlled. Gender was included because male and female EMR 
children are often treated differentially. We hasten to point out that ^^^^^^ 
attraction measures are the jDarariieter estimates obta^ined by fitting, a multi- 
variate model to each data matrix. Thus each individual EMR child's 
attractiveness parameter is adjusted for the structure of the group he or she was 
in. ,In terms of the evaluative mddd w;e proposed earHer, only the socval 
structure (pre- and post-) arid persorial attributes were observed: whereas the 
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indicator variable for treatment can be thought of as a sunogate for temporal 
opportanity and task: 

The regression obtained an adjusted r of .26 using 28 observations. The 
coefficients for pretreatment attraction, treatment, and gender were .78, .83, 
and .94, respectively, with r-statistics of 3:13, I;79, and 1.86. We are 
concerned pnmaniy with the results for treatment since we view pretreatment 
attraction and gender as nuisance parameters. The nhding of a positive efject for 
ireatment with a ^-statistic of 1.79 lends support to the hypothesis that the 
treatment did have a beneficial impact on the average net attractiveness of the 
EMR children. Since the treatment variable appears in the analysis as a 0/1 
indi'^ator variable, when the child is in a control group, pbsttreatmeht attraction " 
appears to depend on pretreatment attraction and jj.ehder. Being in the treatment 
group gives an additional boost to the child*s attraction. 



Concluding Remarks 

In this chapter we have proposed a conceptual framework that focuses 
attention on how interventions alter opportunities for interaction: We also 
proposed a methodology, involving the observation of social activities, which 
we believe can help make distinctions between social and academic outcomes of 
educational innovations clearer: We have argued that social relationship data is 
an especially useful means of obtaining information on the effects of social 
processes ihat are involved in educational interventions. Past problems in the 
analysis of relational data seem to have been solved and newly available 
techniques allow investigators to obtain empirical estimates of structural 
parameters in multivariate models. This results in significantly improved 
precisic '1 and validity of measurement Using data from a maihstreamihg 
experiment we showed that this approach was essential in evaluating treatment 
effectiveness and demonstrated how conceptual mcxiels of programmatically 
altered opportunities could lead to a focus on specific outcomes that, in turn, 
could be associated with particular parameters of a stochastic structural 
model. 

We have tried to capitalize oh some of the advances that have been made in 
the area oT evaluating acadeniic outcomes and bring them to bear bri the 
problem of evaluating social ihtehehtioris. We have argued that a useful 
evaluation needs to have a clear focus bh at least brie valued butcbriie and that 
that outcome should be measureable-. For the sbcial interventibris described, we 
otTer the pattern of sbcial relations as a reasbriable bUtcbme and sociometric 
data as a useful way of riiieasuririg that outcome. We have also argued that the^ 
pblicy arialysis arid evaluative dialogues are greatly enhanced by making 
explicit the uriderlyirig causal schenia through which an intervention is 
presuriied tb operate. It is the evaluatbr's responsibility to develop such a 
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schema and we suggest that evaluators can combine their fcribwledge of social 
science uith an opportunities framework in order to produce such a schema. 
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